Systems and methods for disambiguating a voice search query

ABSTRACT

Systems and methods are described herein for disambiguating a voice search query that contains a command keyword by determining whether the user spoke a quotation from a content item and whether the user mimicked or approximated the way the quotation is spoken in the content item. The voice search query is transcribed into a string, and an audio signature of the voice search query is identified. Metadata of a quotation matching the string is retrieved from a database that includes audio signature information for the string as spoken within the content item. The audio signature of the voice search query is compared with the audio signature information in the metadata to determine whether the audio signature matches the audio signature information in the quotation metadata. If a match is detected, then a search result comprising an identifier of the content item from which the quotation comes is generated.

BACKGROUND

The present disclosure relates to providing search results and, moreparticularly, disambiguation of a voice search query based on audioproperties of the voice input.

SUMMARY

Voice search applications on content platforms allow the user to searchfor content using voice commands. Using command keywords in conjunctionwith search parameters, the user can instruct the application to performa search query for particular content items. Users can also use a famousquote from a particular content item as a search query for that contentitem. When quotes contain command keywords, however, the application maynot recognize that the user is attempting to search for the particularcontent item from which the quote comes and instead performs a searchusing the words of the quote that follow the recognized command keyword.

Systems and methods are described herein for disambiguating a voicesearch query that contains a command keyword by determining whether theuser spoke a quotation from a content item and whether the user mimickedor approximated the way the quotation is spoken in the content item. Ifso, a search result comprising an identifier of the content item isgenerated. The voice search query may also be processed based on thecommand keyword, which may return a number of search results. The searchresult representing the content item from which the quotation comes maybe ranked highest among the search results returned and thereforepresented first in a list of search results. If the user did not mimicor approximate the way the quotation is spoken in the content time, thena search result may not be generated for the content item or may beranked lowest among other search results.

Upon receiving the voice search query, the system transcribes the voicesearch query into a string of text. An audio signature of the voicesearch query is also identified. A query is made to a database ofquotations using the string. Metadata of a quotation matching the stringis received in response to the query. The metadata includes audiosignature information for the string as spoken within the content itemand an identifier of the content item from which the quotation comes.The audio signature of the voice search query is compared with the audiosignature information in the metadata of the content item, and thesystem determines whether the audio signature of the voice search querymatches the audio signature information in the quotation metadata. If amatch is detected, then a search result comprising an identifier of thecontent item from which the quotation comes is generated. The system mayestablish upper and lower thresholds of similarity by transposing theaudio signature information in a positive direction and negativedirection, respectively, by a predetermined amount. The systemdetermines that the audio signature matches the audio signatureinformation if the audio signature falls between the upper and lowerthresholds.

As part of determining whether the audio signature of the voice searchquery matches the audio signature information in the quotation metadata,a cadence of each word in the voice search query may be determined. Thesystem compares the cadence of each word with cadence information in themetadata to determine whether the voice search query was spoken withidentical or similar cadence to that with which the words are spoken inthe content item. If so, the system determines that the audio signaturematches the audio signature information in the quotation metadata. Thesystem may establish upper and lower thresholds of similarity bymodulating the cadence information in a positive direction and anegative direction, respectively, by a predetermined amount. The systemdetermines that the cadence of each word matches the cadence informationfor the corresponding word if the cadence falls between the upper andlower thresholds.

The system may, alternatively or additionally, analyze emphasis placedon each word in the voice search query and compare the relative emphasisof each word with relative emphasis information in the quotationmetadata. If the relative emphasis placed on each word of the voicesearch query is identical or similar to that with which the words arespoken in the content item, the system determines that the audiosignature matches the audio signature information in the quotationmetadata.

The system may also analyze the duration of each word in the voicesearch query and compare the duration of each word with durationinformation of each word in the quotation metadata. If the duration ofeach word is identical or similar to that of each word as spoken in thecontent item, the system determines that the audio signature matches theaudio signature information in the quotation metadata. The system mayestablish upper and lower thresholds of similarity increasing anddecreasing the duration information, respectively, by a predeterminedamount. The system determines that the duration matches the durationinformation if the duration falls between the upper and lowerthresholds.

The overall rhythm of the words spoken in the voice search query mayalso be analyzed and compared with rhythm information in the metadata ofa content item. If the rhythm is identical or similar to the rhythminformation, the system determines that the audio signature matches theaudio signature information in the quotation metadata. The system mayestablish upper and lower thresholds of similarity by modulating therhythm information in a positive direction and a negative direction,respectively, by a predetermined amount. The system determines that therhythm matches the rhythm information in the quotation metadata if therhythm falls between the upper and lower thresholds.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects and advantages of the disclosure will beapparent upon consideration of the following detailed description, takenin conjunction with the accompanying drawings, in which like referencecharacters refer to like parts throughout, and in which:

FIG. 1 shows an exemplary search interface, in accordance with someembodiments of the disclosure;

FIG. 2 shows another exemplary search interface, in accordance with someembodiments of the disclosure;

FIG. 3 shows exemplary voice search data and quotation metadata to becompared, in accordance with some embodiments of the disclosure;

FIG. 4 is a block diagram representing control circuitry, components,and data flow therebetween for disambiguating a voice search query, inaccordance with some embodiments of the disclosure;

FIG. 5 is a flowchart representing a process for disambiguating a voicesearch query, in accordance with some embodiments of the disclosure;

FIG. 6 is a flowchart representing a process for determining whether anaudio signature matches audio signature information, in accordance withsome embodiments of the disclosure;

FIG. 7 is a flowchart representing a second process for determiningwhether an audio signature matches audio signature information, inaccordance with some embodiments of the disclosure;

FIG. 8 is a flowchart representing a third process for determiningwhether an audio signature matches audio signature information, inaccordance with some embodiments of the disclosure;

FIG. 9 is a flowchart representing a fourth process for determiningwhether an audio signature matches audio signature information, inaccordance with some embodiments of the disclosure;

FIG. 10 is a flowchart representing a fifth process for determiningwhether an audio signature matches audio signature information, inaccordance with some embodiments of the disclosure;

FIG. 11 is a flowchart representing a process for determining whether acadence of each word of a plurality of words matches cadence informationfor each corresponding word of a second plurality of words, inaccordance with some embodiments of the disclosure;

FIG. 12 is a flowchart representing a process for determining whether arelative emphasis of each word of a plurality of words matches relativeemphasis information for each corresponding word of a second pluralityof words, in accordance with some embodiments of the disclosure;

FIG. 13 is a flowchart representing a process for determining whether aduration of each word of a plurality of words matches durationinformation for each corresponding word of a second plurality of words,in accordance with some embodiments of the disclosure; and

FIG. 14 is a flowchart representing a process for determining whetherthe overall rhythm of a plurality of words matches rhythm informationfor a second plurality of words, in accordance with some embodiments ofthe disclosure.

DETAILED DESCRIPTION

Voice-based search applications are optimized for natural languageinput. Certain words or phrases are designated as command keywords,which inform the application of what function the user wants toactivate. For example, in natural language, the user may say “Show me”or “I want to see,” followed by the title of a particular content itemor a description of the type of content the user wishes to search for.The application therefore designates “Show me” and “I want to see” ascommand keywords indicating that the application should perform a searchon the words that immediately follow the command keyword. Thus, if theuser says “Show me the money,” the application will search for contententitled “the money.” However, the user may say “Show me the money” in away that mimics how actors Tom Cruise and Cuba Gooding Jr. say thephrase in the movie “Jerry Maguire,” which indicates that the user wantsto see that movie. The application compares an audio signature of thevoice search with audio signature information of content items withknown quotations matching the words of the voice search. If the audiosignature of the voice search is the same as or similar to the audiosignature information of a content item, the application generates asearch result for that particular content item. In some embodiments, theapplication may assign a confidence score to the content item andperform a regular content search based on the voice input as well,assigning a confidence score to each resulting content item. Theapplication then generates search results for the content items havingthe highest confidence scores. If the audio signature of the voicesearch is different from the audio signature information or is extremelyconsistent throughout, the application may assign a lowest confidencescore to the content item from which the quotation comes.

FIG. 1 shows an exemplary search interface 100, according to someembodiments of the disclosure. The search application receives the voicesearch query 102 comprising the words “Show me the money!” Theapplication transcribes the voice search query 102 into a string of text104 (displayed in a truncated fashion in FIG. 1). The applicationqueries a database for content items matching text 104, and contentitems with known quotations matching or similar to text 104. Theapplication receives, in response to the query, metadata of at least onequotation. Included in the metadata of each quotation is audio signatureinformation for the quotation. The application compares an audiosignature of the voice search query with the audio signature informationin the metadata of each quotation. To account for variations in uservocal patterns, and accuracy of user memory and mimicry of thequotation, the application may establish upper and lower thresholds ofsimilarity. The upper threshold is established by scaling, modulating,and/or transposing parameters of the audio signature information in apositive direction. For example, increasing the pitch, tone, or speed ofthe audio signature. Similarly, the lower threshold is established byscaling, modulating, and/or transposing the parameters of the audiosignature in a negative direction. The application then determineswhether each corresponding parameter of the audio signature of the voicesearch query is between the upper and lower thresholds. If so, theapplication determines that the voice search query comprises thequotation and generates, as the first result of a plurality of searchresults 106, a search result 108 comprising an identifier of the contentitem from which the quotation comes. For example, if the audio signatureof the voice search query indicates that the user said “Show me themoney!” in a way that is similar to how actors Tom Cruise and CubaGooding Jr. say the phrase in the movie “Jerry Maguire,” the applicationgenerates a search result 108 for the movie “Jerry Maguire.” Theapplication also generates for display a still image 110 from the movieof a scene in which the quotation is said, as well as a thumbnail image112 representing the movie and summary information 114 describing themovie. Search results may be ordered based on rank, where higher ranksare associated with matches closer to the search string. In someembodiments, search result 108 may be ranked highest among all searchresults. If the audio signature of the voice search query indicates thatthe user did not say the quotation in a way that is similar to how theactors say the phrase, a search result for “Jerry Maguire” may not begenerated, or may be ranked lower among the search results.

FIG. 2 shows another exemplary search interface presented on a mobiledevice 200, in accordance with some embodiments of the disclosure.Mobile device 200 receives voice search query 102 and displays thetranscribed text of the voice search query in search box 202. Athumbnail image 204 of the movie “Jerry Maguire” is displayed as thefirst search result in response to voice search query 102.

FIG. 3 shows exemplary voice search data and quotation metadata to becompared, in accordance with some embodiments of the disclosure. Voicesearch data 300 represents the text 302 and audio signature parameters304, 306, 308, 310, and 312 of the voice search query 102. Volumeparameter 304 represents a scaled volume level at which the user spokethe voice search query. The volume level may be scaled based on anabsolute decibel level or by comparing the volume level with an averagevolume level at which the user speaks based on a voice profile of theuser. Tone parameter 306 indicates an emotion corresponding to the wayin which the user spoke the voice query. The tone may be determined byanalyzing the volume, speed, emphasis and other factors related to howthe user spoke the voice search query.

Word emphasis parameter 308 indicates a relative emphasis of each wordof the transcribed string 302. Continuing the above example, the string“show me the money” contains four words, and the emphasis placed on eachword is represented by a corresponding entry 308 a, 308 b, 308 c, and308 d in word emphasis parameter 308. Similarly, word duration parameter310 indicates the duration of each word, for example in milliseconds,with each word of the transcribed string 302 having a correspondingentry 310 a, 310 b, 310 c, and 310 d in word duration parameter 310.

The application may compare the audio signature to a vocal profile ofthe user to determine whether the user has spoken the voice search queryusing an altered voice. For example, the user may attempt to mimic thesound of an Tom Cruise's voice when saying “Show me the money.” Thealtered voice parameter 312 indicates whether the user has altered hisor her voice when speaking the voice search query.

Quotation metadata 314 represents information about the quotation,including string information 316, volume parameter 318, tone parameter320, word emphasis parameter 322 including entries 322 a, 322 b, 322 c,and 322 d, and word duration parameter 324 including entries 324 a, 324b, 324 c, and 324 d. The application compares these parameters with thecorresponding parameters of the voice search data 300 to determinewhether the user said the quotation. Quotation metadata 314 alsoincludes source identifier 326 identifying the content item from whichthe quotation comes.

FIG. 4 is a block diagram representing control circuitry, components,and data flow therebetween for disambiguating a voice search query, inaccordance with some embodiments of the disclosure. Voice input 400(e.g., voice search query 102) is received using input circuitry 402.Input circuitry 402 may be a data interface such as a Bluetooth module,WiFi module, or other suitable data interface through which audio datacaptured by another device can be received. Alternatively, inputcircuitry 402 may be a microphone through which audio information iscaptured directly. Input circuitry 402 may convert the audio to adigital format such as WAV. Input circuitry 402 communicates voice input400 to control circuitry 404. Control circuitry 404 may be based on oneor more microprocessors, microcontrollers, digital signal processors,programmable logic devices, field-programmable gate arrays (FPGAs),application specific integrated circuits (ASICs), etc., and may includea multi-core processor (e.g., dual-core, quad-core, hexa-core, or anysuitable number of cores) or supercomputer. In some embodiments,processing circuitry may be distributed across multiple separateprocessors or processing units, for example, multiple of the same typeof processing units (e.g., two Intel Core i7 processors) or multipledifferent processors (e.g., an Intel Core i5 processor and an Intel Corei7 processor). Input circuitry 402 transfers 406 voice input 400 totranscription circuitry 408 of control circuitry 404. Transcriptioncircuitry 408 comprises speech-to-text circuitry and/or programmingwhich transcribes voice input 400 into a string of text (e.g., text 104,string 302). Input circuitry 402 also communicates 410 voice input 400to comparison circuitry 412 of control circuitry 404. Comparisoncircuitry 412 compares the audio signature of voice input 400 with audiosignature information in metadata of at least one content item.

Transcription circuitry 408 transfers 414 the string to transceivercircuitry 416. Transceiver circuitry 416 may be a network connectionsuch as an Ethernet port, WiFi module, or any other data connectionsuitable for communicating with a remote server. Transceiver circuitrytransmits 418 a query to quotation database 420 for quotations whichmatch the string.

The query may be an SQL “SELECT” command, or any other suitable queryformat. Transceiver circuitry 416 receives 422, in response to thequery, quotation metadata from quotation database 420. Transceivercircuitry 416 transfers 424 the quotation metadata to comparisoncircuitry 412. Comparison circuitry 412 compares the audio signature ofthe voice input 400 with audio signature information in the quotationmetadata. Control circuitry 404 may establish upper and lower thresholdsof similarity for the audio signature as described above. Comparisoncircuitry 412 may determine whether the audio signature of the voiceinput 400 falls between the upper and lower thresholds. If comparisoncircuitry 412 determines that the audio signature of voice input 400matches audio signature information corresponding to a particularcontent item, comparison circuitry 412 transmits 426 a signal to outputcircuitry 428 to generate for display a content recommendationcomprising an identifier of the particular content item. Outputcircuitry 428, which may be a GPU, VGA port, HDMI port, or any othersuitable graphical output component, then generates for display 430 asearch result comprising an identifier of the particular content item.

FIG. 5 is a flowchart representing an illustrative process 500 fordisambiguating a voice search query, in accordance with some embodimentsof the disclosure. Process 500 may be implemented on control circuitry404. In addition, one or more actions of process 500 may be incorporatedinto or combined with one or more actions of any other process orembodiment described herein.

At 502, control circuitry 404 receives, from input circuitry 402, avoice search query. At 504, control circuitry 404, using transcriptioncircuitry 408, transcribes the voice search query into a stringcomprising a plurality of words. Transcription circuitry 408 may use anysuitable text-to-speech technique to transcribe the voice search query.

At 506, control circuitry 404 determines an audio signature of the voicesearch query. Control circuitry 404 analyzes the voice search query toidentify audio parameters of the voice search query including tone,pitch, word emphasis, word duration, and volume. For example, controlcircuitry 404 may analyze the wave amplitude of the voice search queryto determine the volume at which the user spoke the voice search query.Control circuitry 404 may analyze frequency data of the voice searchquery to determine the pitch and tone of the voice search query. Controlcircuitry 404 may isolate a portion of the voice search querycorresponding to each of the plurality of words and determine theduration of each portion. Control circuitry 404 may compare eachisolated portion of the voice search query to determine the relativelevel of emphasis placed on each word by the user.

At 508, control circuitry 404 queries the content database with thestring. For example, control circuitry 404 may construct and transmit anSQL “SELECT” command to the content database to retrieve quotationmetadata of all quotations matching the string, or significant portionsthereof. At 510, control circuitry 404 receives, in response to thequery, metadata of a plurality of quotations. The metadata includes asaudio signature information of the quotation and an identifier of thecontent item from which the quotation comes.

At 512, control circuitry 404 initializes a counter variable N and setsits value to 0. At 514, control circuitry 404 determines whether theaudio signature of the voice search query matches the audio signatureinformation in the metadata of the N^(th) quotation. Methods fordetermining whether the audio signature is a match are described belowin connection with FIGS. 6-10. If the audio signature of the voicesearch query matches the audio signature information in the metadata ofthe N^(th) quotation then, at 516, control circuitry 404, using outputcircuitry 428, generates for display a search result comprising anidentifier of the content item from which the N^(th) quotation comes.

If the audio signature of the voice search query does not match theaudio signature information in the metadata of the N^(th) quotation, orafter generating a search result for the N^(th) quotation, then, at 518,control circuitry 404 determines whether there is metadata of additionalquotations available. If so, then, at 520, control circuitry incrementsthe value of counter variable N by one, and processing returns to step514.

The actions and descriptions of FIG. 5 may be used with any otherembodiment of this disclosure. In addition, the actions and descriptionsdescribed in relation to FIG. 5 may be done in suitable alternativeorders or in parallel to further the purposes of this disclosure.

FIG. 6 is a flowchart representing an illustrative process 600 fordetermining whether an audio signature matches audio signatureinformation, in accordance with some embodiments of the disclosure.Process 600 may be implemented on control circuitry 404. In addition,one or more actions of process 600 may be incorporated into or combinedwith one or more actions of any other process or embodiment describedherein.

A cadence of a word can be identified by tracking changes in pitch overtime as the word spoken. At 602, control circuitry 404 determines acadence of each word of the plurality of words. For each word, controlcircuitry 404 identifies a cadence of the word by analyzing changes inpitch during pronunciation of the word. Control circuitry 404 generatesa data structure that tracks the changes in pitch over time for eachword.

At 604, control circuitry 404 retrieves metadata of at least onequotation having a string that is similar to the words of the voicesearch query. The metadata also includes cadence information for eachword of the string.

At 606, control circuitry 404 initializes a counter variable N with avalue of 0, a second counter variable C with a value of 0, and avariable T with a value equal to the number of words in the string. At608, control circuitry determines whether the cadence of the N^(th) wordmatches the cadence information of the corresponding word in themetadata. Methods for determining whether the cadence matches thecadence information are described below in connection with FIG. 11. Ifthe cadence of the N^(th) word matches the cadence information of thecorresponding word in the metadata then, at 610, control circuitryincrements the value of counter variable C by one. After incrementingthe value of C, or if the cadence of the N^(th) word does not match thecadence information, at 612, control circuitry compares N with T todetermine whether there are additional words for which cadenceinformation is to be compared. If N is not equal to T, then, at 614,control circuitry 404 increments the value of N by one and processingreturns to step 608. If N is equal to T, meaning that the cadence ofeach word of the voice search query has been compared with cadenceinformation of the corresponding word in the metadata of a quotation,then, at 616, control circuitry determines whether the value of countervariable C, representing the number of words having matching cadences,is equal to T, meaning that the cadences of all words match the cadenceinformation. If so, then, at 618, control circuitry 404 determines thatthe audio signature of the voice search matches the audio signatureinformation in the metadata of the quotation.

The actions and descriptions of FIG. 6 may be used with any otherembodiment of this disclosure. In addition, the actions and descriptionsdescribed in relation to FIG. 6 may be done in suitable alternativeorders or in parallel to further the purposes of this disclosure.

FIG. 7 is a flowchart representing a second illustrative process 700 fordetermining whether an audio signature matches audio signatureinformation, in accordance with some embodiments of the disclosure.Process 700 may be implemented on control circuitry 404. In addition,one or more actions of process 700 may be incorporated into or combinedwith one or more actions of any other process or embodiment describedherein.

At 702, control circuitry 404 determines an emphasis of each word of theplurality of words. For each word, control circuitry 404 analyzes thevolume of the word relative to other words of the plurality of words.Control circuitry 404 generates a data structure that tracks the changesin volume over time for the plurality of words.

At 704, control circuitry 404 retrieves metadata of at least onequotation having a string that is similar to the words of the voicesearch query. The metadata also includes emphasis information for eachword of the string.

At 706, control circuitry 404 initializes a counter variable N with avalue of 0, a second counter variable C with a value of 0, and avariable T with a value equal to the number of words in the string. At708, control circuitry determines whether the emphasis of the N^(th)word matches the emphasis information of the corresponding word in themetadata. Methods for determining whether the emphasis matches theemphasis information are described below in connection with FIG. 12. Ifthe emphasis of the N^(th) word matches the emphasis information of thecorresponding word in the metadata then, at 710, control circuitry 404increments the value of counter variable C by one. After incrementingthe value of C, or if the emphasis of the N^(th) word does not match theemphasis information, at 712, control circuitry 404 compares N with T todetermine whether there are additional words for which emphasisinformation is to be compared. If N is not equal to T then, at 714,control circuitry 404 increments the value of N by one and processingreturns to step 708. If N is equal to T, meaning that the emphasis ofeach word of the voice search query has been compared with emphasisinformation of the corresponding word in the metadata of a quotation,then, at 716, control circuitry 404 determines whether the value ofcounter variable C, representing the number of words having matchingemphases, is equal to T, meaning that the emphases of all words matchthe emphasis information. If so then, at 718, control circuitry 404determines that the audio signature of the voice search matches theaudio signature information in the metadata of the quotation.

The actions and descriptions of FIG. 7 may be used with any otherembodiment of this disclosure. In addition, the actions and descriptionsdescribed in relation to FIG. 7 may be done in suitable alternativeorders or in parallel to further the purposes of this disclosure.

FIG. 8 is a flowchart representing a third illustrative process 800 fordetermining whether an audio signature matches audio signatureinformation, in accordance with some embodiments of the disclosure.Process 800 may be implemented on control circuitry 404. In addition,one or more actions of process 800 may be incorporated into or combinedwith one or more actions of any other process or embodiment describedherein.

At 802, control circuitry 404 determines a duration of each word of theplurality of word. For each word, control circuitry 404 identifies theamount of time taken for the word. Control circuitry 404 generates adata structure that tracks the duration of each word.

At 804, control circuitry 404 retrieves metadata of at least one contentitem having a string that is similar to the words of the voice searchquery. The metadata also includes duration information for each word ofthe string.

At 806, control circuitry 404 initializes a counter variable N with avalue of 0, a second counter variable C with a value of 0, and avariable T with a value equal to the number of words in the string. At808, control circuitry determines whether the duration of the N^(th)word matches the duration information of the corresponding word in themetadata. Methods for determining whether the duration matches theduration information are described below in connection with FIG. 13. Ifthe duration of the N^(th) word matches the duration information of thecorresponding word in the metadata, then, at 810, control circuitry 404increments the value of counter variable C by one. After incrementingthe value of C, or if the duration of the N^(th) word does not match theduration information, at 812, control circuitry 404 compares N with T todetermine whether there are additional words for which durationinformation is to be compared. If N is not equal to T, then, at 814,control circuitry 404 increments the value of N by one and processingreturns to step 808. If N is equal to T, meaning that the duration ofeach word of the voice search query has been compared with durationinformation of the corresponding word in the metadata of a quotation,then, at 816, control circuitry 404 determines whether the value ofcounter variable C, representing the number of words having matchingdurations, is equal to T, meaning that the durations of all words matchthe duration information. If so, then, at 818, control circuitry 404determines that the audio signature of the voice search matches theaudio signature information in the metadata of the quotation.

The actions and descriptions of FIG. 8 may be used with any otherembodiment of this disclosure. In addition, the actions and descriptionsdescribed in relation to FIG. 8 may be done in suitable alternativeorders or in parallel to further the purposes of this disclosure.

FIG. 9 is a flowchart representing a fourth illustrative process 900 fordetermining whether an audio signature matches audio signatureinformation, in accordance with some embodiments of the disclosure.Process 900 may be implemented on control circuitry 404. In addition,one or more actions of process 900 may be incorporated into or combinedwith one or more actions of any other process or embodiment describedherein.

At 902, control circuitry 404 determines an overall rhythm of theplurality of words. Control circuitry 404 analyzes word duration andword emphasis together to identify a rhythm of the plurality of words.Control circuitry 404 generates a data structure that tracks the rhythmof the plurality of words. At 904, control circuitry 404 retrievesmetadata of at least one quotation having a string that is similar tothe words of the voice search query. The metadata also includes rhythminformation for the string.

At 906, control circuitry determines whether the overall rhythm of theplurality of words matches the rhythm information. Methods fordetermining whether the rhythm matches the rhythm information aredescribed below in connection with FIG. 14. If control circuitry 404determines that the overall rhythm of the voice search query matches therhythm information, then, at 908, control circuitry 404 determines thatthe audio signature of the voice search query matches the audiosignature information in the metadata of the quotation.

The actions and descriptions of FIG. 9 may be used with any otherembodiment of this disclosure. In addition, the actions and descriptionsdescribed in relation to FIG. 9 may be done in suitable alternativeorders or in parallel to further the purposes of this disclosure.

FIG. 10 is a flowchart representing a fifth illustrative process 1000for determining whether an audio signature matches audio signatureinformation, in accordance with some embodiments of the disclosure.Process 1000 may be implemented on control circuitry 404. In addition,one or more actions of process 1000 may be incorporated into or combinedwith one or more actions of any other process or embodiment describedherein.

At 1002, control circuitry 404 establishes a lower audio signaturesimilarity threshold by negatively transposing (i.e., decrease the valueof each parameter) the audio signature information in the metadata of aquotation by a predetermined amount. For example, control circuitry mayreduce the volume, word duration, emphasis, pitch, tone, and otherparameters comprising the audio signature by 25%. At 1004, controlcircuitry 404 similarly establishes an upper audio signature similaritythreshold by positively transposing (i.e., increasing the value of eachparameter) the audio signature information in the metadata of aquotation by the same predetermined amount.

At 1006, control circuitry 404 determines whether the audio signature ofthe voice search query is between the lower and upper thresholds.Control circuitry 404 may analyze the audio signature as a whole or maydetermine whether each individual parameter is between the correspondingparameter value of the lower and upper thresholds. If the audiosignature of the voice search query is between the lower and upperthresholds, then, at 1008, control circuitry 404 determines that theaudio signature matches the audio signature information in the metadataof the quotation.

The actions and descriptions of FIG. 10 may be used with any otherembodiment of this disclosure. In addition, the actions and descriptionsdescribed in relation to FIG. 10 may be done in suitable alternativeorders or in parallel to further the purposes of this disclosure.

FIG. 11 is a flowchart representing an illustrative process 1100 fordetermining whether a cadence of each word of a plurality of wordsmatches cadence information for each corresponding word of a secondplurality of words, in accordance with some embodiments of thedisclosure. Process 1100 may be implemented on control circuitry 404. Inaddition, one or more actions of process 1100 may be incorporated intoor combined with one or more actions of any other process or embodimentdescribed herein.

At 1102, control circuitry 404 establishes a lower cadence similaritythreshold by negatively modulating the cadence information for each wordin the metadata of a quotation by a predetermined amount. As describedabove, cadence information is represented by a change in pitch over timefor each word. Control circuitry 404 modulates the change in pitch to asmaller change in pitch by a predetermined amount, such as 25%.Similarly, at 1104, control circuitry 404 establishes an upper cadencesimilarity threshold by positively modulating the cadence informationfor each word by the predetermined amount.

At 1106, control circuitry 404 initializes a counter variable N with avalue of 0, a second counter variable C with a value of 0, and avariable T with a value equal to the number of words in the string. At1108, control circuitry determines whether the cadence of the N^(th)word is between the upper and lower cadence similarity thresholds of thecorresponding word in the metadata. If the cadence of the N^(th) word isbetween the upper and lower thresholds of the corresponding word in themetadata then, at 1110, control circuitry 404 increments the value ofcounter variable C by one. After incrementing the value of C, or if thecadence of the N^(th) word is not between the upper and lower thresholdsthen, at 1112, control circuitry 404 compares N with T to determinewhether there are additional words for which cadence information is tobe compared. If N is not equal to T then, at 1114, control circuitry 404increments the value of N by one and processing returns to step 1108. IfN is equal to T, meaning that the cadence of each word of the voicesearch query has been compared with the upper and lower thresholds ofthe corresponding words in the metadata of a quotation, then, at 1116,control circuitry 404 determines whether the value of counter variableC, representing the number of words having cadences between theirrespective upper and lower thresholds, is equal to T, meaning that thecadences of all words are between their respective upper and lowerthresholds. If so, then, at 1118, control circuitry 404 determines thatthe cadence of each word matches the cadence information for eachcorresponding word in the metadata of the quotation.

The actions and descriptions of FIG. 11 may be used with any otherembodiment of this disclosure. In addition, the actions and descriptionsdescribed in relation to FIG. 11 may be done in suitable alternativeorders or in parallel to further the purposes of this disclosure.

FIG. 12 is a flowchart representing an illustrative process 1200 fordetermining whether a relative emphasis of each word of a plurality ofwords matches relative emphasis information for each corresponding wordof a second plurality of words, in accordance with some embodiments ofthe disclosure. Process 1200 may be implemented on control circuitry404. In addition, one or more actions of process 1200 may beincorporated into or combined with one or more actions of any otherprocess or embodiment described herein.

At 1202, control circuitry 404 determines a first plurality of relativeemphasis levels corresponding to the relative emphasis between each wordof the plurality of words. As discussed above, control circuitry 404analyzes a volume of each word of the voice search query and determinesa relative volume level for each word, with louder words having greateremphasis. At 1204, control circuitry 404 determines a second pluralityof relative emphasis levels between each word in the metadata of aquotation. Control circuitry 404 may retrieve emphasis information fromthe metadata or determine relative emphasis levels for each word basedon other audio information such as volume information in the metadata ofthe quotation.

At 1206, control circuitry 404 initializes a counter variable N with avalue of 0, a second counter variable C with a value of 0, and avariable T with a value equal to the number of words in the string. At1208, control circuitry determines whether the N^(th) relative emphasislevel of the first plurality of emphasis levels is within a thresholdamount of the N^(th) relative emphasis level of the second plurality ofemphasis levels. For example, control circuitry 404 determines whetherthe change in relative emphasis level between two words in the voicesearch query is within 25% of the change in relative emphasis levelbetween the same two words in the metadata of the quotation. If theN^(th) relative emphasis level of the first plurality of emphasis levelsis within the threshold amount of the N^(th) relative emphasis level ofthe second plurality of emphasis levels, then, at 1210, controlcircuitry 404 increments the value of counter variable C by one. Afterincrementing the value of C, or if the cadence of the N^(th) word is notbetween the upper and lower thresholds then, at 1212, control circuitry404 compares N with T to determine whether there are additional wordsfor which cadence information is to be compared. If N is not equal to T,then, at 1214, control circuitry 404 increments the value of N by oneand processing returns to step 1208. If N is equal to T, meaning thatthe each relative emphasis level of the first plurality of emphasislevels is within the threshold amount of the corresponding emphasislevels of the second plurality of emphasis levels, then, at 1216,control circuitry 404 determines whether the value of counter variableC, representing the number of relative emphasis levels of the firstplurality of emphasis levels within the threshold amount of thecorresponding emphasis levels of the second plurality of emphasislevels, is equal to T. If so, then, at 1218, control circuitry 404determines that the emphasis of each word matches the emphasisinformation for each corresponding word in the metadata of thequotation.

The actions and descriptions of FIG. 12 may be used with any otherembodiment of this disclosure. In addition, the actions and descriptionsdescribed in relation to FIG. 12 may be done in suitable alternativeorders or in parallel to further the purposes of this disclosure.

FIG. 13 is a flowchart representing an illustrative process 1300 fordetermining whether a duration of each word of a plurality of wordsmatches duration information for each corresponding word of a secondplurality of words, in accordance with some embodiments of thedisclosure. Process 1300 may be implemented on control circuitry 404. Inaddition, one or more actions of process 1300 may be incorporated intoor combined with one or more actions of any other process or embodimentdescribed herein.

At 1302, control circuitry 404 establishes a lower duration similaritythreshold by reducing the duration information for each word in themetadata of a quotation by a predetermined amount, such as 25%.Similarly, at 1304, control circuitry 404 establishes an upper durationsimilarity threshold by increasing the duration information for eachword by the predetermined amount.

At 1306, control circuitry 404 initializes a counter variable N with avalue of 0, a second counter variable C with a value of 0, and avariable T with a value equal to the number of words in the string. At1308, control circuitry determines whether the duration of the N^(th)word is between the upper and lower duration similarity thresholds ofthe corresponding word in the metadata. If the duration of the N^(th)word is between the upper and lower thresholds of the corresponding wordin the metadata then, at 1310, control circuitry 404 increments thevalue of counter variable C by one. After incrementing the value of C,or if the duration of the N^(th) word is not between the upper and lowerthresholds, then, at 1312, control circuitry 404 compares N with T todetermine whether there are additional words for which durationinformation is to be compared. If N is not equal to T, then, at 1114,control circuitry 404 increments the value of N by one and processingreturns to step 1108. If N is equal to T, meaning that the duration ofeach word of the voice search query has been compared with the upper andlower thresholds of the corresponding words in the metadata of aquotation, then, at 1316, control circuitry 404 determines whether thevalue of counter variable C, representing the number of words havingdurations between their respective upper and lower thresholds, is equalto T, meaning that the durations of all words are between theirrespective upper and lower thresholds. If so, then, at 1318, controlcircuitry 404 determines that the duration of each word matches theduration information for each corresponding word in the metadata of thequotation.

The actions and descriptions of FIG. 13 may be used with any otherembodiment of this disclosure. In addition, the actions and descriptionsdescribed in relation to FIG. 13 may be done in suitable alternativeorders or in parallel to further the purposes of this disclosure.

FIG. 14 is a flowchart representing an illustrative process 1400 fordetermining whether the overall rhythm of a plurality of words matchesrhythm information for a second plurality of words, in accordance withsome embodiments of the disclosure. Process 1400 may be implemented oncontrol circuitry 404. In addition, one or more actions of process 1400may be incorporated into or combined with one or more actions of anyother process or embodiment described herein.

At 1402, control circuitry 404 establishes a lower rhythm similaritythreshold by negatively modulating the rhythm information by apredetermined amount, such as 25%. For example, control circuitry 404may reduce the duration and relative emphasis parameters of each word bythe predetermined amount. Similarly, at 1404, control circuitry 404establishes an upper rhythm similarity threshold by positivelymodulating the rhythm information by the predetermined amount.

At 1406, control circuitry 404 determines whether the overall rhythm ofthe voice search query is between the upper and lower rhythm similaritythresholds. Control circuitry 404 may compare the rhythm as a whole, oreach individual rhythm parameter, with the upper and lower thresholds.If the overall rhythm is between the upper and lower thresholds, then,at 1408, control circuitry 404 determines that the overall rhythm of theplurality of words matches the rhythm information in the metadata of thequotation.

The actions and descriptions of FIG. 14 may be used with any otherembodiment of this disclosure. In addition, the actions and descriptionsdescribed in relation to FIG. 14 may be done in suitable alternativeorders or in parallel to further the purposes of this disclosure.

The processes described above are intended to be illustrative and notlimiting. One skilled in the art would appreciate that the steps of theprocesses discussed herein may be omitted, modified, combined, and/orrearranged, and any additional steps may be performed without departingfrom the scope of the invention. More generally, the above disclosure ismeant to be exemplary and not limiting. Only the claims that follow aremeant to set bounds as to what the present invention includes.Furthermore, it should be noted that the features and limitationsdescribed in any one embodiment may be applied to any other embodimentherein, and flowcharts or examples relating to one embodiment may becombined with any other embodiment in a suitable manner, done indifferent orders, or done in parallel. In addition, the systems andmethods described herein may be performed in real time. It should alsobe noted that the systems and/or methods described above may be appliedto, or used in accordance with, other systems and/or methods.

1-51. (canceled)
 52. A method comprising: determining an audio signatureof a voice search query, wherein the audio signature comprises aplurality of audio characteristics; retrieving, in response to the voicesearch query, metadata of a quotation, wherein the metadata comprises aplurality of audio characteristics characterized by audio signatureinformation corresponding to the quotation and an identifier of acontent item comprising the quotation; comparing the audio signature ofthe voice search query to the audio signature information in themetadata of the quotation; determining, based on the comparing, whethera portion of the plurality of audio characteristics of the audiosignature of the voice search query match a corresponding portion of theplurality of audio characteristics characterized by the audio signatureinformation; and in response to determining the portion of the pluralityof audio characteristics of the audio signature of the voice searchquery matches the corresponding portion of the plurality of audiocharacteristics characterized by the audio signature information,generating for display a search result comprising the identifier of thecontent item comprising the quotation.
 53. The method of claim 52,wherein the audio signature comprises a plurality of audiocharacteristics and a plurality of corresponding values for each of theplurality of audio characteristics.
 54. The method of claim 53, whereinthe plurality of audio characteristics comprise at least one of acadence, a spacing of each word spoken relative to other words, arhythm, and a duration of each word spoken.
 55. The method of claim 52,wherein the plurality of audio characteristics characterized by audiosignature information comprises at least one of a cadence, a spacing ofeach word spoken relative to other words, a rhythm, and a duration ofeach word spoken.
 56. The method of claim 52, further comprising:retrieving a similarity threshold value for each of the plurality ofaudio characteristics, wherein the similarity threshold valuecorresponds to an allowable difference between a value for each of theplurality of audio characteristics of the audio signature of the voicesearch query and a value for each of the plurality of audiocharacteristics characterized by audio signature informationcorresponding to the quotation; computing a difference between a valuecorresponding to the audio signature of the voice search query to avalue corresponding to the audio signature information in the metadataof the quotation; comparing a difference between values corresponding toeach of the characteristic the audio signature of the voice search queryto a corresponding characteristic of the audio signature information inthe metadata of the quotation; and in response to determining adifference between values corresponding to each of the characteristicthe audio signature of the voice search query to the correspondingcharacteristic of the audio signature information in the metadata of thequotation is the same as or is below the similarity threshold value,determining, based on the comparing, that the audio signature of thevoice search query matches the audio signature information correspondingto the quotation.
 57. The method of claim 56, further comprising:establishing a lower threshold by negatively transposing the audiosignature information in the metadata by a predetermined amount;establishing an upper threshold by positively transposing the audiosignature information in the metadata by the predetermined amount; anddetermining whether the audio signature of the voice search query isbetween the lower threshold and the upper threshold.
 58. The method ofclaim 52, wherein determining, based on the comparing, whether theportion of the plurality of audio characteristics of the audio signatureof the voice search query match the corresponding portion of theplurality of audio characteristics characterized by the audio signatureinformation comprises: establishing a lower threshold by negativelymodulating cadence information for each word of the voice search queryby a predetermined amount; establishing an upper threshold by positivelymodulating the cadence information for each word of the voice searchquery by the predetermined amount; and determining whether the cadenceof each word of the voice search query is between the lower thresholdand the upper threshold for a corresponding word of the quotation. 59.The method of claim 52, wherein determining, based on the comparing,whether the portion of the plurality of audio characteristics of theaudio signature of the voice search query match the correspondingportion of the plurality of audio characteristics characterized by theaudio signature information comprises: determining a first plurality ofrelative emphasis levels corresponding to a relative emphasis betweeneach word of the voice search query; determining a second plurality ofrelative emphasis levels corresponding to the relative emphasis betweeneach word of the quotation; and determining, for each relative emphasislevel of the first plurality of relative emphasis levels, whether therespective relative emphasis level is within a threshold amount of thecorresponding relative emphasis level of the second plurality ofemphasis levels.
 60. The method of claim 52, wherein determining, basedon the comparing, whether the portion of the plurality of audiocharacteristics of the audio signature of the voice search query matchthe corresponding portion of the plurality of audio characteristicscharacterized by the audio signature information comprises:establishing, for each word of the quotation, a lower threshold durationby reducing the duration information by a predetermined amount;establishing, for each word of the quotation, an upper thresholdduration by increasing the duration information by the predeterminedamount; and determining, for each word of the voice search query,whether the duration of each respective word is between the lowerthreshold duration and the upper threshold duration for thecorresponding word of the quotation.
 61. The method of claim 52, whereindetermining, based on the comparing, whether the portion of theplurality of audio characteristics of the audio signature of the voicesearch query match the corresponding portion of the plurality of audiocharacteristics characterized by the audio signature informationcomprises: establishing a lower threshold rhythm by negativelymodulating rhythm information corresponding to the quotation by apredetermined amount; establishing an upper threshold rhythm bypositively modulating the rhythm information by the predeterminedamount; and determining whether the overall rhythm of the voice searchquery is between the lower threshold rhythm and the upper thresholdrhythm.
 62. A system comprising: memory; and control circuitryconfigured to: determine an audio signature of a voice search query,wherein the audio signature comprises a plurality of audiocharacteristics; retrieve, in response to the voice search query,metadata of a quotation, wherein the metadata comprises a plurality ofaudio characteristics characterized by audio signature informationcorresponding to the quotation and an identifier of a content itemcomprising the quotation; compare the audio signature of the voicesearch query to the audio signature information in the metadata of thequotation; determine, based on the comparing, whether a portion of theplurality of audio characteristics of the audio signature of the voicesearch query match a corresponding portion of the plurality of audiocharacteristics characterized by the audio signature information; and inresponse to determining the portion of the plurality of audiocharacteristics of the audio signature of the voice search query matchesthe corresponding portion of the plurality of audio characteristicscharacterized by the audio signature information, generate for display asearch result comprising the identifier of the content item comprisingthe quotation.
 63. The system of claim 62, wherein the control circuitryis configured to determine that the audio signature comprises aplurality of audio characteristics and a plurality of correspondingvalues for each of the plurality of audio characteristics.
 64. Thesystem of claim 63, wherein the control circuitry is further configuredto determine the plurality of audio characteristics comprise at leastone of a cadence, a spacing of each word spoken relative to other words,a rhythm, and a duration of each word spoken.
 65. The system of claim62, wherein the control circuitry is further configured to determine theplurality of audio characteristics characterized by audio signatureinformation comprises at least one of a cadence, a spacing of each wordspoken relative to other words, a rhythm, and a duration of each wordspoken.
 66. The system of claim 62, wherein the control circuitry isfurther configured to: retrieve a similarity threshold value for each ofthe plurality of audio characteristics, wherein the similarity thresholdvalue corresponds to an allowable difference between a value for each ofthe plurality of audio characteristics of the audio signature of thevoice search query and a value for each of the plurality of audiocharacteristics characterized by audio signature informationcorresponding to the quotation; compute a difference between a valuecorresponding to the audio signature of the voice search query to avalue corresponding to the audio signature information in the metadataof the quotation; compare a difference between values corresponding toeach of the characteristic the audio signature of the voice search queryto a corresponding characteristic of the audio signature information inthe metadata of the quotation; and in response to determining adifference between values corresponding to each of the characteristicthe audio signature of the voice search query to the correspondingcharacteristic of the audio signature information in the metadata of thequotation is the same as or is below the similarity threshold value,determine, based on the comparing, that the audio signature of the voicesearch query matches the audio signature information corresponding tothe quotation.
 67. The system of claim 66, wherein the control circuitryis further configured to: establish a lower threshold by negativelytransposing the audio signature information in the metadata by apredetermined amount; establish an upper threshold by positivelytransposing the audio signature information in the metadata by thepredetermined amount; and determine whether the audio signature of thevoice search query is between the lower threshold and the upperthreshold.
 68. The system of claim 62, wherein the control circuitryconfigured to determine, based on the comparing, whether the portion ofthe plurality of audio characteristics of the audio signature of thevoice search query match the corresponding portion of the plurality ofaudio characteristics characterized by the audio signature informationis further configured to: establish a lower threshold by negativelymodulating cadence information for each word of the voice search queryby a predetermined amount; establish an upper threshold by positivelymodulating the cadence information for each word of the voice searchquery by the predetermined amount; and determine whether the cadence ofeach word of the voice search query is between the lower threshold andthe upper threshold for a corresponding word of the quotation.
 69. Thesystem of claim 62, wherein the control circuitry configured todetermine, based on the comparing, whether the portion of the pluralityof audio characteristics of the audio signature of the voice searchquery match the corresponding portion of the plurality of audiocharacteristics characterized by the audio signature information isfurther configured to: determine a first plurality of relative emphasislevels corresponding to a relative emphasis between each word of thevoice search query; determine a second plurality of relative emphasislevels corresponding to the relative emphasis between each word of thequotation; and determine, for each relative emphasis level of the firstplurality of relative emphasis levels, whether the respective relativeemphasis level is within a threshold amount of the correspondingrelative emphasis level of the second plurality of emphasis levels. 70.The system of claim 62, wherein the control circuitry configured todetermine, based on the comparing, whether the portion of the pluralityof audio characteristics of the audio signature of the voice searchquery match the corresponding portion of the plurality of audiocharacteristics characterized by the audio signature information isfurther configured to: establish, for each word of the quotation, alower threshold duration by reducing the duration information by apredetermined amount; establish, for each word of the quotation, anupper threshold duration by increasing the duration information by thepredetermined amount; and determine, for each word of the voice searchquery, whether the duration of each respective word is between the lowerthreshold duration and the upper threshold duration for thecorresponding word of the quotation.
 71. The system of claim 62, whereinthe control circuitry configured to determine, based on the comparing,whether the portion of the plurality of audio characteristics of theaudio signature of the voice search query match the correspondingportion of the plurality of audio characteristics characterized by theaudio signature information is further configured to: establish a lowerthreshold rhythm by negatively modulating rhythm informationcorresponding to the quotation by a predetermined amount; establish anupper threshold rhythm by positively modulating the rhythm informationby the predetermined amount; and determine whether the overall rhythm ofthe voice search query is between the lower threshold rhythm and theupper threshold rhythm.